Methodology
- Import data
- EDA
- Check Null Values & use appropriate fillers
- Check the distribution of Class variable
- Implement a Label Encoder
- Check the variables distribution- Bivariate Analysis
- Check the Correlation between the variables
- Prepare X and Y
- Split the data into Train and Test
- Train an SVM and get accuracy
- Use PCA and capture 95% of the variance in the data
- Find the accuracy of the model using PCA
- Compare the accuracy of SVM using raw data and SVM using PCA
- Scale X
- Repeat steps 9-13 above using the scaled X
In [1]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.pyplot import figure
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')
In [2]:
#reading the data
data = pd.read_csv("vehicle.csv")
In [3]:
data.head() #checking the head
Out[3]:
In [4]:
data.shape #checking the shape of the data
Out[4]:
In [5]:
data.describe().T
Out[5]:
In [6]:
data.dtypes
Out[6]:
In [7]:
data.skew()
Out[7]:
Observations¶
- There are a total of 846 records with 19 columns
- From describe the count is not 846 for all the columns and this indicates missing values
- 18 columns are of numeric type while the class column is of type object
- max.length_aspect_ratio , pr.axis_aspect_ratio, scaled_radius_of_gyration.1 are highly skewed while the other columns are marginally skewed. hollows_ratio is negatively skewed
In [8]:
# Checking for missing data
data.isnull().sum()
Out[8]:
Observations¶
Many of the variables have missing data except compactness, max.length_aspect_ratio,max.length_rectangularity,hollows_ratio and class
In [9]:
# checking the median of the data
data.median()
Out[9]:
In [10]:
#dealing with missing data
newData = data # to keep the original df constant, introducing a new df
#define a median filler
medianFiller = lambda x: x.fillna(x.median())
#apply the filler
newData.iloc[:,0:18] = data.iloc[:,0:18].apply(medianFiller)
In [11]:
#checking for missing data in the new dataframe
newData.isnull().sum()
Out[11]:
Observations¶
All the missing values have been replaced and it can be seen the data frame has no missing values.
Understanding the class variable¶
In [12]:
newData['class'].value_counts()
Out[12]:
In [13]:
newData['class'].value_counts(1)*100
Out[13]:
Observations¶
- There are three classes or types of vehicles Car, Bus and Van
- The distribution is Car approx 50%, Bus approx 25%, Van approx 23%
Next Steps¶
Since the class is an object variable now, it will be difficult to see the distribution of other columns with respect to the class and hence it is required to be converted into a numeric type using an Encoder.
In [14]:
newData.dtypes # checking the datatype before encoding
Out[14]:
In [15]:
# using a Label Encoder
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
data_le = newData # encoded df
data_le['class'] = le.fit_transform(data_le['class'])
In [16]:
data_le.dtypes # checking the df after encoding
Out[16]:
In [17]:
data_le['class'].value_counts()
Out[17]:
In [18]:
sns.pairplot(data_le, hue = 'class')
Out[18]:
Observations¶
Listing out only a few observations
- Bus, Van, and Car are represented by blue, green, orange colors on the graph
- scaled_radius_of_gyration.1, pr.axis_aspect_ratio, max.length_aspect_ratio, radius_ratio have significant outliers as seen from the distribution
- max.length_aspect_ratio has multiple peaks in its distribution similar to scaled_variance.1
- a few of the features have a good correlation as observed from the plots, skewness_about.2 vs hollows_ratio,skewness_about.1 vs hollows_ratio,
- Elongatedness and scaled_variance.1 has a very strong negative correlation
- Most of the other distributions have outliers
In [19]:
# Checking for Correlation
corr = data_le.corr()
mask = np.zeros_like(corr, dtype = np.bool)
mask[np.triu_indices_from(mask)] = True
f, ax = plt.subplots(figsize = (15, 15))
cmap = sns.diverging_palette(220, 10, as_cmap = True)
sns.heatmap(corr, mask = mask, cmap = cmap, vmax = 1, center = 0, square = True,
linewidths = .5, cbar_kws = {"shrink": .5}, annot = True)
Out[19]:
Observations¶
- Only 5 features have a positive correlation with class and none of them show a real strong correlation
- 13 features are negatively correlated in various degrees of correlation
- Elongatedness, hollows_ratio, max.length_aspect_ratio are fairly correlated in order of correlation and can be considered alone for buliding base model.
- At the same time, hollows_ratio has a strong correlation with skewness_about.2, fair correlation with radius_ratio and compactness and hence even these can be considered in building the model.
- Also, max.length_aspect_ratio has a fairly strong correlation with pr.axis_aspect_ratio, followed by low correlation with distance_circularity, circularity and compactness.
So it is observed there is a fair bit of correlation among the features and this will drive the inclusion of most of them to build a base model.
Version 1 - Dropping only the class column¶
In [20]:
# preparing data - dropping only the class column
from sklearn.model_selection import train_test_split
X = data_le.drop(columns = ['class']) # Predictor feature columns
Y = data_le['class'] # Predicted class
In [21]:
#preparing the training and testing data set
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, random_state=25)
#check the data split
print("{0:0.2f}% data is in training set".format((len(x_train)/len(data_le.index)) * 100))
print("{0:0.2f}% data is in test set".format((len(x_test)/len(data_le.index)) * 100))
In [22]:
#SVM - non scaled version
from sklearn import svm
from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score, roc_curve, accuracy_score
SVM = svm.SVC(random_state=25)
SVM.fit(x_train, y_train)
SVM_train_predict = SVM.predict(x_train)
SVM_test_predict = SVM.predict(x_test)
print('SVM accuracy for train set: {0:.3f}'.format(SVM.score(x_train, y_train)))
print('SVM accuracy for test set: {0:.3f}'.format(SVM.score(x_test, y_test)))
# Classification Report
print('\n{}'.format(classification_report(y_test, SVM_test_predict)))
# Accuracy Score
auc = accuracy_score(y_test, SVM_test_predict)
print('\nAccuracy Score:', auc.round(3))
In [23]:
# Performing Cross Validation
from sklearn.model_selection import cross_val_score
svm_cvs = cross_val_score(svm.SVC(random_state=25), X, Y)
svm_cvs_mean = svm_cvs.mean()
svm_cvs_mean
Out[23]:
Version 2 - Dropping class and other non-influencing columns¶
In [24]:
# preparing data - dropping columns which doesn't seem to influence the predictor variable
Xnew = data_le.drop(columns = ['class', 'skewness_about', 'skewness_about.1', 'skewness_about.2', 'scaled_radius_of_gyration.1']) # Predictor feature columns
Ynew = data_le['class'] # Predicted class
In [25]:
#preparing the training and testing data set
x_trainN, x_testN, y_trainN, y_testN = train_test_split(Xnew, Ynew, test_size=0.3, random_state=25)
#check the data split
print("{0:0.2f}% data is in training set".format((len(x_trainN)/len(data_le.index)) * 100))
print("{0:0.2f}% data is in test set".format((len(x_testN)/len(data_le.index)) * 100))
In [26]:
#SVM - non scaled version and columns dropped
from sklearn import svm
from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score, roc_curve, accuracy_score
SVMN = svm.SVC(random_state=25)
SVMN.fit(x_trainN, y_trainN)
SVMN_train_predict = SVMN.predict(x_trainN)
SVMN_test_predict = SVMN.predict(x_testN)
print('SVM accuracy for train set: {0:.3f}'.format(SVMN.score(x_trainN, y_trainN)))
print('SVM accuracy for test set: {0:.3f}'.format(SVMN.score(x_testN, y_testN)))
# Classification Report
print('\n{}'.format(classification_report(y_testN, SVMN_test_predict)))
# Accuracy Score
aucNew = accuracy_score(y_testN, SVMN_test_predict)
print('\nAccuracy Score:', aucNew.round(3))
In [27]:
# Performing Cross Validation
svmN_cvs = cross_val_score(svm.SVC(random_state=25), Xnew, Ynew)
svmN_cvs_mean = svmN_cvs.mean()
svmN_cvs_mean
Out[27]:
Observations¶
There is no significant improvement in the accuracy of the model atter dropping more features.
Performing PCA - non scaled features¶
In [28]:
from sklearn.decomposition import PCA
pca = PCA(n_components=18) # no. of components = no. of features
pca.fit(X)
# print(pca.explained_variance_)
# print(pca.components_)
Out[28]:
In [29]:
print(pca.explained_variance_ratio_)
In [30]:
plt.bar(list(range(1,19)),pca.explained_variance_ratio_,alpha=0.5, align='center')
plt.ylabel('Variation explained')
plt.xlabel('Eigen Value')
plt.show()
In [31]:
plt.step(list(range(1,19)),np.cumsum(pca.explained_variance_ratio_), where='mid')
plt.ylabel('Cum of variation explained')
plt.xlabel('eigen Value')
plt.show()
Observations¶
It can be observed the first component accounts for more than 95% of the variance and hence only one component will be considered further for buliding the model.
In [32]:
pca = PCA(n_components=1)
pca.fit(X)
# print(pca.components_)
# print(pca.explained_variance_ratio_)
Xpca = pca.transform(X)
Fitting an SVM using the Principal Components¶
In [34]:
# using principal components to fit an SVM model
model_pca = svm.SVC(random_state=25)
model_pca.fit(Xpca, Y)
model_pca.score(Xpca, Y)
Out[34]:
Version 3 - Scaling the features and fitting an SVM¶
In [35]:
# scaling the features
from scipy.stats import zscore
XScaled=X.apply(zscore)
XScaled.head()
Out[35]:
In [36]:
#preparing the training and testing data set for scaled data
x_trainS, x_testS, y_trainS, y_testS = train_test_split(XScaled, Y, test_size=0.3, random_state=25)
#check the data split
print("{0:0.2f}% data is in training set".format((len(x_trainS)/len(data_le.index)) * 100))
print("{0:0.2f}% data is in test set".format((len(x_testS)/len(data_le.index)) * 100))
In [37]:
#SVM - scaled version
SVMScaled = svm.SVC(random_state=25)
SVMScaled.fit(x_trainS, y_trainS)
SVM_train_predictS = SVMScaled.predict(x_trainS)
SVM_test_predictS = SVMScaled.predict(x_testS)
print('SVM accuracy for train set: {0:.3f}'.format(SVMScaled.score(x_trainS, y_trainS)))
print('SVM accuracy for test set: {0:.3f}'.format(SVMScaled.score(x_testS, y_testS)))
# Classification Report
print('\n{}'.format(classification_report(y_testS, SVM_test_predictS)))
# Accuracy Score
aucS = accuracy_score(y_testS, SVM_test_predictS)
print('\nAccuracy Score:', aucS.round(3))
In [38]:
# Performing Cross Validation
svmS_cvs = cross_val_score(svm.SVC(random_state=25), XScaled, Y)
svmS_cvs_mean = svmS_cvs.mean()
svmS_cvs_mean
Out[38]:
Performing PCA - Scaled features¶
In [39]:
pcaS = PCA(n_components=18)
pcaS.fit(XScaled)
Out[39]:
In [40]:
plt.bar(list(range(1,19)),pcaS.explained_variance_ratio_,alpha=0.5, align='center')
plt.ylabel('Variation explained')
plt.xlabel('eigen Value')
plt.show()
In [41]:
plt.step(list(range(1,19)),np.cumsum(pcaS.explained_variance_ratio_), where='mid')
plt.ylabel('Cum of variation explained')
plt.xlabel('eigen Value')
plt.show()
In [42]:
# Considering 6 components which represent close to 95% of the variance
pcaS = PCA(n_components=6)
pcaS.fit(XScaled)
#print(pcaS.components_)
print(pcaS.explained_variance_ratio_)
In [43]:
XpcaS = pcaS.transform(XScaled)
In [44]:
model_pcaS = svm.SVC(random_state=25)
model_pcaS.fit(XpcaS, Y)
model_pcaS.score(XpcaS, Y)
Out[44]:
Final Observations and Conclusions¶
Checking the accuracies on the Test data set
- Accuracy of Base SVM model: $0.69$
- Accuracy of Base SVM Model + Cross Validation: $0.68$
- Accuracy of Base SVM Model + PCA: $0.63$
- Accuracy of Base SVM Model + Scaled Features: $0.97$
- Accuracy of Base SVM Model + Scaled Features + Cross Validation: $0.96$
- Accuracy of Base SVM Model + Scaled Features + PCA: $0.906$
By dropping the dimensions to 1 for the non-scaled model the accuracy has reduced by 5% while in the case of scaled features by dropping the dimensions to 6 there is a drop of around 6% in the accuracy. The scaled model perfomed well and has a good test accuracy of around 90% with PCA using the SVM model.
In [ ]: